1. Field of the Invention
The present invention is directed to the distribution of operations of a source program among a plurality of cooperating processors and more particularly to a system for managing the distribution of operations from unadorned source code with none or limited programmer interaction.
2. Description of the Related Art
In the past many decades, programmers have been writing applications for computers that have progressed from simple, single instruction pointer, linear operation processors to concurrent, multi-processor environments. Current and future computers will have multiple execution units, multiple cores or multiple processors. This is partly because the basic CPU clock frequency will soon hit an upper limit due to problems related to the physics of manufacturing silicon-based semiconductors. Since the invention of the microprocessor around 1970, the complexity (e.g., number of transistors) and clock speed of the microprocessor has continually increased. These increases came about by shrinking the size of the individual transistors within the microprocessor, thereby reducing the capacitance across the transistor and allowing them to switch faster. Currently, a major CPU manufacturer is producing processors with 65 nm geometries. The creation of the microprocessor involves photolithography on a silicon substrate. Unfortunately, due to the size of the light wave passing through the photo mask, there is a limit on how small the mask can be before the mask blocks all light at a given wavelength. Even though doubling of clock speed has not continued over the last few silicon generations, chip die size has continued to increase, providing vast transistor budgets for the chip designer. This has led processor manufacturers into developing processors that have multiple execution units to perform faster multiple tasks in parallel, thereby improving the execution performance of a program at the same or similar clock speeds. The newer generation of processors multi-thread (or hyper thread) or have separate discrete processors that cooperate to execute a single task. One such type of multi-core processor is known as a cell-based processor, in which the processing system is comprised of a large number of independent processors or cells that communicate with each other and accept or pass tasks and results to each other.
Programming languages have also progressed at a rapid rate. In the early years, computers were programmed using panels of wire jumpers or by programming the computer in its machine language. Assemblers were created allowing programmers to program the computer with programs stored on, for example, paper tape, and written in assembly language. The assemblers would then convert program into the machine language for execution. Soon, higher level languages such as FORTRAN, C and PL/1 further removed the programmer from having to know and understand the underlying hardware. Furthermore, specialized languages for specific tasks such as PERL and LISP were created to handle processing needs for certain disciplines such as system administration, system artificial intelligence and simulation/modeling. High level languages are designed to abstract the underlying hardware so that the programmer need not be concerned with the overall architecture of the processing system that will run the resulting executable. Furthermore, high level languages are designed so that the source code can be re-compiled and run on totally different architectures with slight or no changes needed to the source code. This feature makes for portable code, but the ability of the high-level language compilers to optimize the resulting executable program is minimal. Compilers for a given processor architecture may optimize a program by selecting optimal instructions, allocating registers, performing compiler techniques such as loop unrolling and the like. This optimization helps in single processing environments, but falls short in multiple processing environments because the program needs to be divided in a balanced way across several processing elements so that each processing element completes its respective operations at around the same time, thereby not slowing the combined operation down by the slowest link.
Attempts have been made to port existing languages to multiple processor computing systems. To date, these require programmer intervention, in that the programmer needs to understand the underlying processor architecture and direct the program operation onto the various processing elements. This is analogous to the programmer going back to write machine language or assembly language programs, whereby the programmer must understand the underlying architecture and the program is designed to operate exclusively on that architecture. An example of this is SETI (Search for Extra Terrestrial Intelligence), wherein the program is divided into multiple executables enabled for independent execution on most any personal computer so that a small subsection of data and the program is transferred to a “volunteer” personal computer where the program is executed when the personal computer is idle, such as when the screen saver is operating. In this type of application, many processors are performing similar tasks on different slices of a dataset. In this example, a programmer or team designs the program to operate on a pool of clients of a specific architecture such as an industry standard personal computer running one of the Windows® operating systems. The programmer or team of programmers must determine how to sub-divide the program for operation over the pool of processors and must design a front-end that will accept data (in SETI—data from deep-space radio receivers), sub-divide the data, find an available client, transfer the data to the client, wait for the result and combine the results.
Another limitation of using compilers to distribute a program among multiple processors is that the compiler can only evaluate one high level language statement at a time and, for example, dispatching expressions in the a given statement among the multiple processors. Unfortunately, experience has demonstrated that invoking parallelism at such a fine granularity leads to excessive communication overhead compared to the amount of parallel computation achieved.
Another solution is needed that evaluates the program and data as a whole which balances communication/computation tradeoffs before a parallel program decomposition is attempted.
Some programs developed to solve complex problems in fields such as biotechnology or radar analysis require a programmer write an application that performs many independent, yet differing operations. The program may run on a supercomputer comprised of a plurality of processing elements. In some architectures, the processing elements are homogeneous, for example, cell-based architectures. In other architectures, the processing elements are heterogeneous, for example, dedicated super computers having specialized processing elements to perform specific tasks such as Fast-Fourier Transforms. Each operation requires differing resources, thereby requiring the programmer to understand the resource needs of the program as well as the capabilities of various processing units within the supercomputing architecture. Presently, the task of creating software for these architectures requires the programmer to understand the application and the architecture and write specific code to match both. Since this code is specific the target computer system or supercomputer, it is generally difficult to port the program to a different architecture without major redesign. This is analogous to the early years of computers where the programmer had to understand and use the basic machine code of each processor on which they wrote programs. In each of these architectures, a target program, for example a data compression program, must be written and designed for the specific architecture. So being, a target program that compresses video data to fit on a certain media (e.g., Digital Versatile Disk or DVD) would be written by a programmer who understands the architecture and divides the target program's steps between the various processing elements. For example, several processing elements may be assigned to compress a block of video while another processing element may be assigned to decompress that block to feed back error coefficients to the compression steps. Once developed and debugged, the target program may run fine on the target architecture, but when the target architecture becomes obsolete or fails and must be updated to a new architecture, the programmer must re-write much of the target application, porting it to the new architecture.
What is needed is a system that will analyze programs for operation on multiple processing elements and divide the program automatically onto those processing elements for optimal operation, freeing the programmer from becoming an expert regarding the hardware platform.